= Cirodown

A Markup language to write multipage or single page HTML / PDF books, blogs, etc. that is \x[saner][saner] and more powerful than Markdown and Asciidoctor, but still nicer to write than XML and JSON, with reference implementation in JavaScript.

The source for this document is at: https://github.com/cirosantilli/cirodown/blob/master/README.ciro and a rendered version can be seen at: https://cirosantilli.com/cirodown and a live in-browser demo at: https://cirosantilli.com/cirodown/editor[].

\Toc

== Quick start

Live in-browser demo: https://cirosantilli.com/cirodown/editor

Alternatively, you can clone and run it locally with:
``
git clone https://github.com/cirosantilli/cirodown-template
cd cirodown-template
npm install
./build
xdg-open editor.html
``
We want to make that editor amazing later on: \x[editor-with-preview][live browser editor].

Install the NPM package globally and use it from the command line for a quick conversion:
``
npm install -g cirodown
printf 'ab\ncd\n' | cirodown --body-only
``
or https://stackoverflow.com/questions/9679932/how-to-use-package-installed-locally-in-node-modules[install in a single directory and run locally from it]:
``
printf '{}\n' > package.json
npm install --save cirodown
printf 'ab\ncd\n' | npx cirodown --body-only
``

To start a new project with a minimal sane template \x[generate][use `--generate`] and then \x[publish-to-github-pages][publish it to GitHub with `--publish`]:
``
mkdir -p cirodown-test
cd cirodown-test
git init
git remote add origin git@github.com:USERNAME/cirodown-test.git
cirodown --generate
git add .
git commit -m 'First commit'
cirodown --publish
``
Or to start with a more complex template suitable for multifile projects use \x[generate-multifile][use `--generate-multifile`] instead:
``
cirodown --generate-multifile
``

Render all `.ciro` files from the master of this project to separate files:
``
git clone https://github.com/cirosantilli/cirodown
cd cirodown
npm link
npm link cirodown
./cirodown README.ciro
xdg-open index.html
xdg-open not-readme.html
``
or render README.ciro and all \x[includes] into a single HTML page with \x[html-single-page] and view it:
``
./cirodown .
xdg-open index.html
``
or render instead as one fully offline standalone HTML file that can be given directly to readers by adding \x[html-embed] as well:
``
./cirodown --html-embed --html-single-page README.ciro
``
and give `index.html` to the reader.

Try out the JavaScript API with \a[api_hello.js]:
``
npm install cirodown
./api_hello.js
``

The `npm link` commands allow you to make changes to the code without re-installing the package all the time for development! Try hacking \a[cirodown] to see it. Just remember that if you add a new dependency, you must redo the symlinking business:
``
npm install <dependency>
npm link
npm link cirodown
``
Asked if there is a better way at: https://stackoverflow.com/questions/59389027/how-to-interactively-test-the-executable-of-an-npm-node-js-package-during-develo[]. The symlink business can be undone with:
``
npm unlink
rm node_modules/cirodown
``

TODO: create a multifile blog example folder and link from here.

== Design goals

Cirodown is designed entirely to allow writing complex professional HTML and PDF scientific books, blogs, articles and encyclopedias.

Cirodown aims to be the ultimate LaTeX "killer", allowing books to be finally published as either HTML or PDF painlessly (LaTeX being only a backend to PDF generation), supporting out of the box feature such as:
* \x[internal-cross-references][references] to \x[headers], \x[images], etc. with amazing error checking and reporting: never break internal links gain
* KaTeX server side \x[mathematics]
* multi file features out of the box so you don't need a separate wrapper like Jekyll to make a multi-page website:
  * \x[internal-cross-file-references]
  * \x[table-of-contents] that crosses input files
  * `inotifywait` watch rebuild server
  * cross file includes that work as links on multi output file mode, and true includes in single output file mode
  * cross file configuration files to factor out common page parts like headers and footers
* advanced header/ID related features:
  * \x[scope]
* is written in JavaScript and therefore runs natively on the browser to allow live previews
* helps you with the publishing:
  * `./cirodown --publish` helper for single command publishing to the configured target (default GitHub pages): \x[publish]
  * Cirodown tries to deal with media such as images and video intelligently for you, see e.g.: \x[where-to-store-images]

=== Saner

Originally, Cirodown was is meant to be both saner and more powerful than Markdown and Asciidoctor.

But alas, as Ciro started implementing and using it, he started to bring some Markdown \x[insane-macro-shortcuts][insanity he missed back in].

And so this "degraded" slightly into a language slightly saner than Asciidoctor but with an amazing Node.js implementation that makes it better for book writing and website publishing.

But hopefully, having starting from a saner point will still produce a saner end result, e.g. there are sane constructs for every insane one.

It is intended that this will be an acceptable downside as Cirodown will be used primarily large complex content such as books rather than forum posts, and will therefore primarily written either:
* in text editors locally, where users have more features than in random browser textareas
* in a dedicated website that will revolutionize education, and therefore have a good JavaScript editing interface: https://github.com/cirosantilli/write-free-science-books-to-get-famous-website

For example, originally Cirodown had exactly five magic characters, with similar functions as in LaTeX:
* `\` backslash to start a macro, like LaTeX
* `{` and `}`: left and right square brackets to delimit \x[positional-vs-named-arguments][optional macro arguments]
* `[` and `]`: left and right curly braces bracket to start an optional arguments
and double blank newlines for \x[paragraphs][paragraphs] if you are pedantic, but this later degrated into many more with \x[insane-macro-shortcuts].

We would like to have only square brackets for both optional and mandatory to have even less magic characters, but that would make the language difficult to parse for computer and humans. LaTeX was right for once!

This produces a very regular syntax that is easy to learn, including doing:
* arbitrary nesting of elements
* adding arbitrary properties to elements

This sanity also makes the end tail learning curve of the endless edge cases found in Markdown and Asciidoctor disappear.

The language is designed to be philosophically isomorphic to HTML to:
* further reduce the learning curve
* ensure that most of HTML constructs can be reached, including arbitrary nesting

More precisely:
* macro names map to tag names, e.g.: `\\a` to `<a`
* one of the arguments of macros, maps to the content of the HTML element, and the others map to attributes.

  E.g., in a link:
  ``\a[http://example.com][Link text\]``
  the first macro argument:
  ``http://example.com``
  maps to the `href` of `<a`, and the second macro argument:
  ``Link text``
  maps to the internal content of `<a>Link text<>`.

=== More powerful

The \x[saner][high sanity of Cirodown], also makes creating new macro extensions extremely easy and intuitive.

All built-in language features use the exact same API as new extensions, which ensures that the extension API is sane forever.

Markdown is clearly missing many key features such as block attributes and \x[internal-cross-references], and has no standardized extension mechanism.

The "more powerful than Asciidoctor" part is only partially true, since Asciidoctor is very featureful can do basically anything through extensions.

The difference is mostly that Cirodown is completely and entirely focused on making amazing scientific books, and so will have key features for that application out-of-the box, notably:
* amazing header/ToC/ID features including proper error reports: never have a internal broken link or duplicate ID again
* \x[mathematics][server side pre-rendered maths with KaTeX]: all divs and spans are ready, browser only applies CSS, no JavaScript gets executed
* \x[publish]: we take care of website publishing for you out-of-the-box, no need to integrate into an external project like Jekyll

Another advantage over Asciidoctor is that the reference implementation of Cirodown is in JavaScript, and can therefore be used on browser live preview out of the box. Asciidoctor does Transpile to JS with https://github.com/opal/opal[Opal], but who wants to deal with that layer of complexity?

== Paragraphs
{title2=`\P`}

OK, this is too common, so we opted for some \x[insane-macro-shortcuts][insanity] here: double newline is a paragraph!
\CirodownExample[[
Paragraph 1.

Paragraph 2.
]]

Equivalently however, you can use an explicit `\P` macros as well, which is required for example to add properties to a paragraph, e.g.:
\CirodownExample[[
\P{id=paragraph-1}[Paragraph 1]
\P{id=paragraph-2}[Paragraph 2]
]]

Paragraphs are created automatically inside \x[macro-argument-syntax][macro argument] whenever a double newline appears.

Note that Cirodown paragraphs render in HTML as `div` with `class="p"` and not as `p`. This means that you can add basically anything inside them, e.g. a list:
``
My favorite list is:
\Ul[
\li[aa]
\li[bb]
]
because it is simple.
``
which renders as a single paragraph.

One major advantage of this, is that when writing documentation, you often want to keep lists or code blocks inside a given paragraph, so that it is easy to reference the entire paragraph with an ID. Think for example of paragraphs in the C++ standard.

== Links
{title2=`\a`}

\x[insane-macro-shortcuts][Insane] autolink (link text is the same as the link):
\CirodownExample[[
The website http://example.com is cool. See also:

\Q[http://example.com/2]
]]
Exact parsing rules described at: \x[insane-link-parsing-rules]{full}.

Equivalent sane version:
\CirodownExample[[[
The website \a[http://example.com] is cool.

\Q[\a[http://example.com/2]]
]]]

Insane link with custom text:
\CirodownExample[[
The website http://example.com[example.com] is cool.
]]
Equivalent sane version:
\CirodownExample[[
The website \a[http://example.com][example.com] is cool.
]]
If the custom text is empty, an autolink is generated. This is often useful if you want your link to be followed by punctuation:
\CirodownExample[[
The website is really cool: http://example.com[].
]]
This could also be achieved with the sane syntax of course, but this pattern saves a tiny bit of typing.

Link with multiple paragraphs inside it:
\CirodownExample[[
\a[http://example.com][Multiple

paragraphs]
]]

=== Insane link parsing rules

Insane links must be preceded by either:
* the start of the document
* the start of a \x[positional-vs-named-arguments][positional or named argument] (e.g. `\P[http://example.com]` or `Image[xi.png]{description=http://example.com}`) that not marked with `elide_link_only` (which is the case for example for `href` of `a`)
* space ` ` char
* newline `\n` char

The recognized protocols are the ones shown at: \x[known-url-protocols]{full}.

Insane links end when either the end of the document or one of the following characters is found:
* space ` `
* newline `\n`
* open or close square bracket `[` or `]`
* open or close curly braces `{` or `}`

As a consequence, of those rules, it is impossible to have an insane link followed immediately by a period, e.g.:
\CirodownExample[[
Check out this website: http://example.com.
]]
You must instead use the sane syntax for that:
\CirodownExample[[
Check out this website: \a[http://example.com].
]]

If you want your link to include one of the terminating characters, e.g. `]`, all characters can be escaped with a backslash, e.g.:
\CirodownExample[[
Hello http://example.com/\]a\}b\\c\ d world.
]]

Note that the `http://example.com` inside `\a[http://example.com]` only works because we do some post-processing magic that prevents its expansion, otherwise the link would expand twice:
\CirodownExample[[
\P[http://example.com]

\a[http://example.com]
]]
This magic can be observed with \x[help-macros] by seeing that the `href` argument of the `a` macro has the property: 
``
"elide_link_only": true,
``

TODO it is currently impossible to prevent autolinks from expanding, we can either go with an empty macro solution, or with a literal text solution:
\C[[[
\CirodownExample[[
Literal URL \empty[]http://example.com but why?
Literal URL \text[[http://example.com]] but why?
]]
]]]

== Internal cross references
{title2=`\x`}

Every macro in Cirodown can have an optional `id` and many also have a reserved `title` property.

When a macro in the document has a `title` argument but no `id` argument given, get an auto-generated ID from the title: \x[automatic-id-from-title].

For macros that do have an ID, derived from `title` or not, you can write a cross reference to it, e.g.:
\CirodownExample[[
See this \x[internal-cross-references] awesome section.
]]

An explicit link body can be given just as for regular HTTP \x[links] as:
\CirodownExample[[
See this \x[internal-cross-references][ awesome section].
]]

=== x `child` argument

TODO implement.

Setting `child=1` on a cross reference to a header as in:
``
\xref[my-header]{child=1}
``
has the following effects:
* add the current header to the list of extra parents of the child

This allows a section to have multiple parents, e.g. to include it into multiple categories. For example:
``
= Animals

== Mammals

=== Dog

=== Cat

== Cute animals

Cute animals:
* \xref[dog]{child=1}
* \xref[cat]
``
would render something like:
``
h1 Animals

h2 Mammals

h3 Dog (parent section: Mammals)
(other parent sections: Cute animals)

h3 Cat (parent section: Mammals)

== Cute animals

* \xref[dog]
* \xref[cat]
``
so note how "Dog" has a list of extra parents including "Cute animals", but Cat does not, due to the `child=1`.

This property does not affect how the \x[table-of-contents][`\Toc`] is rendered. We could insert elements sections there multiple times, but it has the downside that browser Ctrl + F searches would hit the same thing multiple times on the table of contents, which might make finding things harder.
``
== My title{id=my-id}

Read this \x[my-id][amazing section].
``

If the second argument, the `content`, is not present, it expand to the header title, e.g.:
``
== My title{id=my-id}

Read this \x[my-id].
``
is the same as:
``
== My title{id=my-id}

Read this \x[my-id][My title].
``

=== Cross reference `full` argument
{id=cross-reference-full-argument}

To also show the section auto-generated number as in "Section X.Y My title" we add the optional `{full}` \x[boolean-named-arguments][boolean named argument] to the cross reference, for example:
\CirodownExample[[
\x[cross-reference-full-argument]{full}.
]]

`{full}` is not needed for cross references to most macros besides \x[headers], which use `full` by default as seen by the `default_x_style_full` macro property in \x[help-macros]. This is for example the case for \x[images]. You can force this to be disabled with `{full=0}`:
\CirodownExample[[
Compare \x[image-my-test-image]{full=0} vs \x[image-my-test-image]{full=1}.
]]

==== Cross reference `full` argument Internal cross file references
{id=cross-reference-full-argument-internal-cross-file-references}

For example in the following \x[internal-cross-file-references][internal cross file reference]:
\CirodownExample[[
\x[h2-in-not-the-readme]{full}.
]]
we get just something like:
``
Section "h2 in not the readme"
``
instead of:
``
Section 1.2 "h2 in not the readme"
``
This is because the number "Section 1.2" might already have been used in the current page, leading to confusion.

==== Cross reference title inflection

A common usage pattern is that we want to use \x[headers]{p=0} titles in \x[cross-reference-full-argument][non-full] \x[internal-cross-references] as the definition of a concept without repeating the title, for example:
``
== Dog

Cute animal.

\x[cats][Cats] are its natural enemies.

== Cats

This is the natural enemy of a \x[dog][dog].

\x[dog][Dogs] are cute, but they are still the enemy.

One example of a cat is \x[felix-the-cat].

=== Felix the Cat

Felix is not really a \x[cats][cat], just a carton character.
``

However, word https://en.wikipedia.org/wiki/Inflection[inflection] makes it much harder to avoid retyping the definition again.

For example, in the previous example, without any further intelligent behaviour we would be forced to re-type `\x[dog][dog]` instead of the desired `\x[dog]`.

Cirodown can take care of some inflection cases for you.

For capitalization, both headers and cross reference macros have the `c` \x[boolean-named-arguments][boolean named argument] which stands for "capitalized":
* for headers, `c` means that the header title has fixed capitalization as given in the title, i.e.
  * if the title has a capital first character, it will always show as a capital, as is the case for most https://en.wikipedia.org/wiki/Proper_noun[proper noun]
  * if it is lower case, it will also always remain lower case, as is the case for some rare proper nouns, notably https://en.wikipedia.org/wiki/Arm_Holdings[the name of certain companies]

  This means that for such headers, `c` in the `x` has no effect. Maybe we should give an error in that case. But lazy now, send PR.
* for cross reference macros, `c` means that the first letter of the title should be capitalized.

  Using this option is required when you are starting a sentence with a non-proper noun.

For pluralization, cross reference macros have the `p` \x[boolean-named-arguments][boolean named argument] which stands for "pluralize":
* if given and true, this automatically pluralizes the last word of the target title by using the https://github.com/blakeembrey/pluralize library
* if given and false, automatically singularize
* if not given, don't change the number of elements
If your desired pluralization is any more complex than modifying the last word of the title, you must do it manually however.

With those rules in mind, the previous Cirodown example can be written with less repetition as:
``
== Dog

Cute animal.

\x[cats]{c} are its natural enemies.

== Cats

This is the natural enemy of a \x[dog].

\x[dog]{p} are cute, but they are still the enemy.

One example of a cat is \x[Felix the Cat].

=== Felix the Cat
{c}

Felix is not really a \x[cats][cat], just a carton character.
``

Now for a live example for quick and dirty interactive testing.

===== Cross reference title inflection example

\CirodownExample[[
\x[inflection-example-not-proper]
]]

\CirodownExample[[
\x[inflection-example-not-proper]{c}
]]

\CirodownExample[[
\x[inflection-example-not-proper]{full}
]]

\CirodownExample[[
\x[inflection-example-proper]
]]

\CirodownExample[[
\x[inflection-example-proper]{c}
]]

\CirodownExample[[
\x[inflection-example-not-proper-lower]
]]

\CirodownExample[[
\x[inflection-example-not-proper-lower]{c}
]]

\CirodownExample[[
\x[inflection-example-proper-lower]
]]

\CirodownExample[[
\x[not-readme]
]]

\CirodownExample[[
\x[not-readme]{c}
]]

\CirodownExample[[
\x[inflection-example-not-proper]{p}
]]

\CirodownExample[[
\x[inflection-plural-examples]
]]

\CirodownExample[[
\x[inflection-plural-examples]{p}
]]

\CirodownExample[[
\x[inflection-plural-examples]{p=0}
]]

\CirodownExample[[
\x[inflection-plural-examples]{p=1}
]]

\CirodownExample[[
\x[not-the-readme-header-with-fixed-case]
]]

====== Inflection example not-proper

====== Inflection example proper
{c}

====== inflection example not-proper lower

====== inflection example proper lower
{c}

====== Inflection plural examples

=== The `title2` argument

The `title2` argument can be given to any element that has the `title` argument.

Its usage is a bit like the `description=` argument of \x[images], allowing you to add some extra content to the header without affecting its ID.

Unlike `description=` however, `title2` shows up on all \x[cross-reference-full-argument][`full`] references, including appearances in the \x[table-of-contents].

For example, given the Cirodown input:
``
The toc:

\Toc

== Quotation
{title2=`\Q`}

== Orthogonal group
{title2=$O(n)$}

\x[quotation]{p}

\x[orthogonal-group]

\x[orthogonal-group]{full}
``
the rendered output looks like:
``
The toc:

* Quotation (\Q)
* Orthogonal group (O(n))

== Quotation (\Q)

== Orthogonal group (O(n))

quotations

orthogonal group

Section 2. "Orthogonal group (O(n))"
``

Parenthesis are added automatically around all rendered `title2`.

=== `\x` within `title` restrictions
{id=x-within-title-restrictions}

If you use `\x` within a `title`, which most commonly happens for \x[images][image titles], that can generate complex dependencies, which would either be harder to implement, or lead to infinite recursion: https://github.com/cirosantilli/cirodown/issues/34

To prevent such problems, Cirodown emits an error if:
* you use an `\x` without explicit `content` argument inside a `\title` that has no ID
* an infinite `x` recursion is detected

For example, consider the code:
``
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{title=That \x[this-my-id3].}

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{title=My id3.}

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{title=This \x[my-id3].}
``

Wee see therefore a complex ID dependency emerge:
* `\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{title=My id3.}` has `my-id3`
* `\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{title=This \x[my-id3].}` renders the title as `This My Id3`, and so would have ID `this-my-id3`
* `\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{title=That \x[this-my-id3].}` can finally render the title as `That This My Id3` and get ID `that-this-my-id3`

While it is technically possible to implement this, it would require a bit of extra work which we don't want to put in right now, and it would be very confusing for end users reading the source anyways.

As a result, the above example currently gives errors, and you could make an equivalent explicit version that has no errors either by giving explicit titles to each image that has an `x` in it (this is normally the saner approach):
``
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=that-this-my-id3}
{title=That \x[this-my-id3].}

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{title=My id3.}

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=this-my-id3}
{title=This \x[my-id3].}
``
or by setting explicit content to every `x` inside a `title` for which an ID wasn't given as in:
``
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=That \x[this-my-id3][This My id3].}

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{title=My id3.}

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=this-my-id3}
{title=This \x[my-id3][My id3].}
``

And for an example of infinite recursion, consider:
``
= \x[cd]
{id=ab}

== \x[ab]
{id=cd}
``

In this example, we note that the `\x` in the first header expands

This circular dependency case if fundamentally unsolvable automatically, the only alternative is for the user to provide an explicit `x` content to at least one of the `x` to break the circular dependency as in either:
``
= \x[cd][cd]
{id=ab}

== \x[ab]
{id=cd}
``
or:
``
= \x[cd]
{id=ab}

== \x[ab][ab]
{id=cd}
``

=== Internal cross file references

Reference to the first header of another file:
\CirodownExample[[
\x[not-readme]
]]

Reference to the first header of another file that is a second inclusion:
\CirodownExample[[
\x[included-by-not-readme]
]]

Reference to another header of another file, with \x[cross-reference-full-argument][`full`]:
\CirodownExample[[
\x[h2-in-not-the-readme]{full}.
]]
Note that when `full` is used with references in another file in \x[html-single-page][multi page mode], the number is not rendered as explained at: \x[cross-reference-full-argument-internal-cross-file-references]{full}.

Reference to an image in another file:
\CirodownExample[[
\x[image-not-readme-xi]{full}.
]]

Reference to an image in another file:
\CirodownExample[[
\x[image-figure-in-not-the-readme-without-explicit-id]{full}.
]]

Remember that the \x[the-toplevel-header][ID of the toplevel header] is automatically derived from its file name, that's why we have to use:
\CirodownExample[[
\x[not-readme]
]]
instead of:
``
\x[not-the-readme]
``

Reference to an internal header of another file: \x[h2-in-not-the-readme]. By default, That header ID gets prefixed by the ID of the top header.

When using \x[html-single-page] mode, the cross file references end up pointing to an ID inside the current HTML element, e.g.:
``
<a href="#not-readme">
``
rather than:
``
<a href="not-readme.html/#not-readme">
``
This is why IDs must be unique for elements across all pages.

==== Internal cross file references internals

When running in Node.js, Cirodown dumps the IDs of all processed files to a `out/ids.sqlite3` file, and then reads from that file when IDs are needed.

When converting a directory, `out/ids.sqlite3` is placed inside the input directory. When converting individual files, `out/ids.sqlite3` is placed in the current working directory. The file is not created or used when handling input from stdin.

When running in the browser, the same JavaScript API will send queries to the server instead of a local SQLite database.

To inspect the ID database to debug it, you can use:
``
sqlite3 out/db.sqlite3 .dump
``

You can force `cirodown` to not use the ID database with `--no-db`:
``
./cirodown --no-db .
``

=== Automatic ID from title

If a \x[the-toplevel-header][non-toplevel] macro has the `title` property is present but no explicit `id`, an ID is created automatically from the `title`, by applying the following transformations:
* convert all of `A-Z` characters to lowercase
* convert consecutive sequences of all non `a-z0-9` ASCII characters to a single hyphen `-`. Note that this leaves non-ASCII character untouched.
* strip leading or trailing hyphens
Note how those rules leave non ASCII Unicode characters untouched, as capitalization and determining if something "is a letter or not" in those cases can be tricky.

So for example, the following automatic IDs would be generated: \x[table-examples-of-automatically-generated-ids].

\Table{title=Examples of automatically generated IDs.}
[
\Tr[
  \Th[title]
  \Th[id]
  \Th[comments]
]
\Tr[
  \Td[My favorite title]
  \Td[my-favorite-title]
  \Td
]
\Tr[
  \Td[Ciro's markdown is awesome]
  \Td[ciro-s-markdown-is-awesome]
  \Td
]
\Tr[
  \Td[The École Polytechnique]
  \Td[the-École-polytechnique]
  \Td[We leave the non ASCII uppercase https://en.wikipedia.org/wiki/Acute_accent[acute accented] `e`, `É`, untouched by default]
]
]

For \x[the-toplevel-header][the toplevel header], its ID is derived from the basename of the Cirodown file without extension instead of from the `title` argument.

== Headers
{title2=`\H`}

\x[insane-macro-shortcuts][Insane] with `= ` (equal sign space):
``
= My h1

== My h2

=== My h3
``
Insane headers end at the first newline found. They cannot therefore contain raw newline tokens.

Equivalent sane:
``
\H[1][My h1]

\H[2][My h2]

\H[3][My h3]
``

Custom ID for \x[internal-cross-references] on insane headers:
``
= My h1
{id=h1}

== My h2
{id=h2}

=== My h3
{id=h3}
``

Sane equivalent:
``
\H[1][My h1]{id=h1}

\H[2][My h2]{id=h2}

\H[3][My h3]{id=h3}
``

=== Id-based header levels
{title2=`{parent=id}`}

In addition to the basic way of specifying header levels with an explicit level number, as mentioned at \x[headers]{full}, Cirodown also supports a more indirect ID-based mechanism.

For example, the following fixed level syntax:
``
= My h1

== My h2 1

== My h2 2

=== My h3 2 1
``
is equivalent to the following ID-based version:
``
= My h1

= My h2 1
{parent=my-h1}

= My h2 2
{parent=my-h1}

= My h3 2 1
{parent=my-h2-h}
``

The main advantages of this syntax are felt when you have a huge document with \x[unlimited-header-levels][very large header depths]. In that case:
* it becomes easy to get levels wrong with so many large level numbers to deal with. It is much harder to get an ID wrong.
* when you want to move headers around to improve organization, things are quite painful without a refactoring tool (which we intend to provide in the \x[editor-with-preview]), as you need to fix up the levels of every single header.

  If you are using the ID-based syntax however, you only have to move the chunk of headers, and change the `parent` argument of a single top-level header being moved.

Note that when the `parent=` argument is given, the header level must be `1`, otherwise Cirodown assumes that something is weird and gives an error. E.g. the following gives an error:
``
= My h1

== My h2
{parent=my-h1}
``
because the second header has level `2` instead of the required `= My h2`.

=== Unlimited header levels

There is no limit to how many levels we can have, for either sane or insane headers!

HTML is randomly limited to `h6`, so Cirodown just renders higher levels as an `h6` with a `data-level` attribute to indicate the actual level for possible CSS styling:
``
<h6 data-level="7">My title</h6>
``

The recommended style is to use insane headers up to `h6`, and then move to sane one for higher levels though, otherwise it becomes very hard to count the `=` signs.

To avoid this, we sondiered making the insane syntax be instead:
``
= 1 My h1
= 2 My h2
= 3 My h3
``
but it just didn't feel as good, and is a bit harder to type than just smashing `=` n times for lower levels, which is the most common use case. So we just copied markdown.

==== My h4

===== My h5

====== My h6

======= My h7

\H[8][My h8]

\H[9][My h9]

\H[10][My h10]

\H[11][My h11]

\H[12][My h12]

\H[13][My h13]

=== Table of contents
{title2=`\Toc`}

Only one ToC shows per document.

Any ToC besides the first one is ignored. TODO make it into an error instead?

The ToC ignores `\H[1]` by default, as we encourage that header level to appear only once and represent the main title under which the entire document goes.

`\Toc` from \x[includes] are ignored.

For when you want a quick outline of the header tree on the terminal, also consider: `--show-headers`.

==== Table of contents JavaScript open close interaction

To the left of table of content entries you can click on an open/close icon to toggle the visibility of different levels of the table of contents.

The expansion algorithm is optimized so that when you first click on the arrow of a node it will give you an overview of that node, showing only its direct children.

At the start, all nodes are showing, to facilitate Ctrl + F queries, and so when you first click on a node, it collapses the children of children, so you can see only the direct children the node of interest, without getting a gazillion of sub descendants shown first.

The exact behaviour is:
* if a node is closed (not showing children), open it (show children), and close all children (don't show children of children)

  This way, we see a summary of the node and direct children only.
* if a node is open:
  * if all chilren are closed, open all children. This allows you to see the full subtree.
  * if all chilren are open, close the node. Except for the toplevel "Table of Contents" node, since we never want to close that one.
  * otherwise (there are both open and closed children), close all children to get an overview of the node

As a result of the above rules, clicking on a node three times goes through a loop of basically all useful states:
* node closed
* node open and children closed
* node open and children open

Clicking on the link from a \x[headers]{p=0} up to the table of contents also automatically opens up the node for you in case it had been previously closed manually.

=== Header `scope` argument
{id=scope}

In some use cases, the sections under a section describe inseparable parts of something.

For example, when documenting an experiment you executed, you will generally want an "Introduction", then a "Materials" section, and then a "Results" section for every experiment.

On their own, those sections don't make much sense: they are always referred to in the context of the given experiment.

The problem is then how to get unique IDs for those sections.

One solution, would be to manually add the experiment ID as prefix to every subsection, as in:
``
= Experiments

See: \x[full-and-unique-experiment-name/materials]

== Introduction

== Full and unique experiment name

=== Introduction
{id=full-and-unique-experiment-name/introduction}

See our awesome results: \x[full-and-unique-experiment-name/results]

For a more general introduction to all experiments, see: \x[introduction].

=== Materials
{id=full-and-unique-experiment-name/materials}

=== Results
{id=full-and-unique-experiment-name/results}
``

but this would be very tedious.

To keep those IDs shorter, Cirodown provides the `scope` \x[boolean-named-arguments][boolean named argument] property of \x[headers], which works analogously to C++ namespaces with the header IDs.

Using `scope`, the previous example could be written more succinctly as:
``
= Experiments

See: \x[full-and-unique-experiment-name/materials]

== Introduction

== Full and unique experiment name
{scope}

=== Introduction

See our awesome results: \x[results]

For a more general introduction to all experiments, see: \x[/introduction].

=== Materials

=== Results
``

Note how:
* full IDs are automatically prefixed by the parent scopes prefixed and joined with a slash `/`
* we can refer to other IDs withing the current scope without duplicating the scope. E.g. `\x[results]` in the example already refers to the ID `full-and-unique-experiment-name/materials`
* to refer to an ID outside of the scope and avoid name conflicts with IDs inside of the current scope, we start a reference with a slash `/`

  So in the example above, `\x[/introduction]` refers to the ID `introduction`, and not `full-and-unique-experiment-name/introduction`.

==== Header `scope` argument of toplevel headers

When \x[the-toplevel-header] is given the `scope` property Cirodown automatically uses the file path for the scope and heaves fragments untouched.

For example, suppose that file `full-and-unique-experiment-name` contains:
``
= Full and unique experiment name
{scope}

== Introduction

== Materials
``

In this case, multi-file output will generate a file called `full-and-unique-experiment-name.html`, and the URL of the subsections will be just:
* `full-and-unique-experiment-name.html#introduction`
* `full-and-unique-experiment-name.html#materials`
instead of
* `full-and-unique-experiment-name.html#full-and-unique-experiment-name/introduction`
* `full-and-unique-experiment-name.html#full-and-unique-experiment-name/materials`

Some quick interactive cross file link tests:
* \x[not-readme-with-scope]
* \x[not-readme-with-scope/h2]
* \x[not-readme-with-scope/image-my-image]

=== Header explicit levels vs nesting design choice

Arguably, the language would be even saner if we did:
``
\H[My h1][

Paragraph.

\H[My h2][]
]
``
rather than having explicit levels.

But we chose not to do it like most markups available because it leads to too many nesting levels, and hard to determine where you are without tooling.

=== Includes
{title2=`\Include`}

The `\Include` macro allows including an external Cirodown headers under the current header.

It exists to allow optional single page HTML output while still retaining the ability to:
* split up large input files into multiple files to make renders faster during document development
* suggest an optional custom output split with one HTML output per Cirodown input, in order to avoid extremely large HTML pages which could be slow to load

`\Include` takes one mandatory argument, which is the relative path without extension to the `.ciro` file that you want to include.

Things would have been a bit more clean if the argument would take the ID of the header you want to include rather than the filename, which is analogous to how \x[internal-cross-references][`\x`] works, but that would mean that a single page build would require an initial parse to determine IDs, so we are just going with the easier option of pointing out the file name directly.

Headers of the included document are automatically shifted to match the level of the child of the level where they are being included.

If \x[html-single-page] is given, the external document is rendered embedded into the current document directly, essentially as if the source had been copy pasted (except for small corrections such as the header offsets).

Otherwise, the following effects happen:
* The headers of the included tree appear in the \x[table-of-contents] of the document as links to the corresponding external files.

  This is implemented simply by reading a previously generated database file much like \x[internal-cross-file-references-internals], which avoids the slowdown of parsing all included files every time.

  As a result, you have to do an initial parse of all files in the project to extract their headers however, just as you would need to do when linking to those headers.
* the include itself renders as a link to the included document
* \x[html-single-page]

Here is an example of inclusion of the files `not-readme.ciro` and `not-readme-2.ciro`.

\Include[not-readme]

\Include[not-readme-2]

\Include[not-readme-with-scope]

=== Skipping header levels

The very first header of a document can be of any level, although we highly recommend your document to start with a `\H[1]`, and to contain exactly just one `\H[1]`, as this has implications such as:
* `\H[1]` is used for the document title: \x[html-document-title]
* `\H[1]` does not show on the \x[table-of-contents]

After the initial header however, you must not skip a header level, e.g. the following would give an error because it skips level 3:
``
= my 1

== my 1

==== my 4
``

=== The toplevel header

If the document has only a single header of the highest level, e.g. like the following has only a single `h2`:
``
== My 2

=== My 3 1

=== My 3 2
``
then this has some magical effects.

==== The toplevel header IDs don't show

Header IDs won't show for the toplevel level. For example, the headers would render like:
``
My 2

1. My 3 1

2. My 3 2
``
rather than:
``
1. My 2

1.2. My 3 1

1.2. My 3 2
``
This is because in this case, we guess that the `h2` is the toplevel.

==== The ID of the first header is derived from the filename

TODO: we kind of wanted this to be the ID of the toplevel header instead of the first header, but this would require an extra postprocessing pass (to determine if the first header is toplevel or not), which might affect performance, so we are not doing it right now.

When the Cirodown input comes from a file (and not e.g. stdin), the default ID of the first header in the document is derived from the basename of the Cirodown input source file rather than from its title.

This is specially relevant when \x[includes][including] other files.

For example, in file named `my-file.ciro` which contains:
``
= Awesome cirodown file
]]
``
the ID of the header is `my-file` rather than `awesome-cirodown-file`. See also: \x[automatic-id-from-title].

If the file is an \x[index-files][index file], then the basename of the parent directory is used instead, e.g. the toplevel ID of a file:
``my-subdir/README.ciro``
would be:
``#my-subdir``
rather than:
``#README.ciro``

== Lists
{title2=`* `, `\L`, `\Ul`, `\Ol`}

\x[insane-macro-shortcuts][Insane] with `* ` (asterisk space):
\CirodownExample[[
* a
* b
* c
]]

Equivalent saner with \x[auto-parent][implicit `ul` container]:
\CirodownExample[[
\L[a]
\L[b]
\L[c]
]]

Equivalent fully sane with explicit container:
\CirodownExample[[
\Ul[
\L[a]
\L[b]
\L[c]
]
]]

The explicit container is required if you want to pass extra arguments properties to the `ul` list macro, e.g. a title and an ID: \x[list-my-id]{full}:
\CirodownExample[[
\Ul
{id=list-my-id}
[
\L[a]
\L[b]
\L[c]
]
]]
This is the case because without the explicit container in an implicit `ul` list, the arguments would stick to the last list item instead of the list itself.

It is also required if you want ordered lists:
\CirodownExample[[
\Ol[
\L[first]
\L[second]
\L[third]
]
]]

Insane nested list with two space indentation:
\CirodownExample[[
* a
  * a1
  * a2
  * a2
* b
* c
]]
The indentation must always be exactly equal to two spaces, anything else leads to errors or unintended output.

Equivalent saner nested lists with implicit containers:
\CirodownExample[[
\L[
a
\L[a1]
\L[a2]
\L[a2]
]
\L[b]
\L[c]
]]

Insane list item with a paragraph inside of it:
\CirodownExample[[
* a
* I have

  Multiple paragraphs.

  * And
  * also
  * a
  * list
* c
]]

Equivalent sane version:
\CirodownExample[[
\L[a]
\L[
I have

Multiple paragraphs.

\L[And]
\L[also]
\L[a]
\L[list]
]
\L[c]
]]

Insane lists may be escaped with a backslash as usual:
\CirodownExample[[
\* paragraph starting with an asterisk.
]]

And now a list outside of \x[cirodown-example] to test how it looks directly under \x[toplevel]:

\L[a]
\L[b]
\L[c]

== Images
{title2=`\Image` and `\image`}

A block image with \x[block-vs-inline-macros][capital] 'i' `Image` showcasing most of the image properties \x[image-my-test-image].
\CirodownExample[[
Have a look at this amazing image: \x[image-my-test-image].

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=The title of my image.}
{id=image-my-test-image}
{width=600}
{height=200}
{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
{description=The description of my image.}
]]
This exemplifies the following parameters:
* `description`: similar to `title`, but allow for further explanations without them appearing in \x[internal-cross-references][cross references] to the image
* `source`: a standardized way to credit an image by linking to a URL that contains further image metadata
For further discussion on the effects of ID see: \x[image-id][full].

And this is how you make an inline image inline one with lower case `i`:
\CirodownExample[[
My inline \image[Tank_man_standing_in_front_of_some_tanks.jpg][test image] is awesome.
]]
Inline images can't have captions.

The `description` can have multiple paragraphs just like any other element, but we don't recommend this style because it makes it hard for readers to see what is part of the image caption and what is part of the next paragraph.
\CirodownExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=image-my-test-image-description-paragraphs}
{title=The title of my image.}
{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
{description=Description paragraph 1.

Description paragraph 2.
}

Paragraph after.
]]
Maybe we could improve this with some CSS styling, but generally when you want multiple paragraphs inside the image, you might instead be better off making an \x[internal-cross-file-references][internal cross reference] to the the image and adding your extended explanation outside.

And now an image outside of \x[cirodown-example] to test how it looks directly under \x[toplevel]:

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=image-my-test-image-toplevel}

=== Image height

By default, we fix image heights to `height=315`, and let the `width` be calculated proportionally once the image loads. We therefore ignore the actual image size. This is done to:
* prevent reflows as the page loads images and can determine their actual sizes, especially is the user opens the page at a given ID in the middle of the page
* create a more uniform media experience by default, unless a custom image size is actually needed e.g. if the image needs to be larger

=== Image IDs and captions
{id=image-id}

Here is an image without a description but with an ID so we can link to it: \x[image-my-test-image-2].
\CirodownExample[[
Have a look at this amazing image: \x[image-my-test-image-2].

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=image-my-test-image-2}
]]
This works because \x[cross-reference-full-argument][`full` is the default cross referene style for `Image`], otherwise the link text would be empty since there is no `title`, and cirodown would raise an error.

Cirodown can optionally deduce the title from the basename of the `src` argument if the `title_from_src` \x[boolean-named-arguments][boolean named argument] is given, or if `title-from-src` is set as the default \x[media-providers][media provider] for the media type: 
\CirodownExample[[
Have a look at this amazing image: \x[image-tank-man-standing-in-front-of-some-tanks].

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title_from_src}
]]

If the image has neither ID nor title, then it does not get a caption at all, and it is not possible to link to it with an \x[internal-cross-references][internal cross reference], e.g.:
\CirodownExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
]]
The image does however get an automatically generated ID based on its image number, and it is possible for readers to link to that ID on the rendered version, e.g. as:
``
#image-123
``
This link is of course not stable across document revisions however, since if an image is added before that one, the link will break. This kind of image is discouraged, because in paged output formats like PDF, it could float away from the text that refers to the image.

We can also see that such an image does not increment the Figure count:
\CirodownExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{id=image-my-test-image-count-before}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{id=image-my-test-image-count-after}
]]

If the image has any visible metadata such as `source` or `description` however, then the caption does show and the Figure count gets incremented:
\CirodownExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{description=This is the description of my image.}
]]

=== Where to store images

==== Store images inside the repository itself

If you are making a limited repository that will not have a ton of images, then you can get away with simply git tracking your images in the main repository.

With this setup, no further action is needed. For example, with a file structure of:
``
./README.ciro
./Tank_man_standing_in_front_of_some_tanks.jpg
``
just use the image from \C[README.ciro] as:
\CirodownExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
]]

However, if you are making a huge tutorial, which can have a huge undefined number of images (i.e. any scientific book), then you likely don't want to git track your images in the git repository.

==== Store images in a separate media repository

In this approach, you create a separate GitHub repository in addition to the main one containing the text to contain only media such as images.

This approach is more suitable than \x[store-images-inside-the-repository-itself] if you are going to have a lot of images.

When using this approach, you could of course just point directly to the final image URL, e.g. as in:
\CirodownExample[[
\Image[https://raw.githubusercontent.com/cirosantilli/media/master/Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]
]]
but Cirodown allows you use configurations that allow you to enter just the image basename: `Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg` which we will cover next.

In order to get this to work, the recommended repository setup is:
* `./main-repo/.git`: main repository at https://github.com/username/main-repo
* `./main-repo/data/media/.git/`: media repository at https://github.com/username/main-repo-media[], and where `data/` is gitignored.
The directory and repository names are not mandatory, but if you place media in `data/media` and name its repository by adding the `*-media` suffix, then `cirodown` will handle everything for you without any further configuration in \x[media-providers].

This particular documentation repository does have a different setup as can be seen from its \a[cirodown.json]. Then, when everything is setup correctly, we can refer to images simply as:
\CirodownExample[[
\Image[Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]{provider=github}
]]
In this example, we also needed to set `{provider=github}` explicitly since it was not set as the default image provider in our `cirodown.json`. In most projects however, all of your images will be in the default repository, so this won't be needed.

`provider` must not be given when a full URL is given because we automatically detect providers from URLs, e.g.:
``
\Image[https://raw.githubusercontent.com/cirosantilli/media/master/Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]{provider=github}
``
is an error.

TODO implement: `cirodown` will even automatically add and push used images in the `my-tutorial-media` repository for you \x[publish][during publishing]!

You should then use the following rules inside `my-tutorial-media`:
* give every file a very descriptive and unique name as a full English sentence
* never ever delete any files, nor change their content, unless it is an improvement in format that does change the information contained of the image TODO link to nice Wikimedia Commons guideline page
This way, even though the repositories are not fully in sync, anyone who clones the latest version of the `*-media` directory will be able to view any version of the main repository.

Then, if one day the media repository ever blows up GitHub's limit, you can just migrate the images to another image server that allows arbitrary basenames, e.g. AWS, and just configure your project to use that new media base URL with the \x[media-providers] option.

The reason why images should be kept in a separate repository is that images are hundreds or thousands of times larger than hand written text.

Therefore, images could easily fill up the maximum repository size you are allowed: https://webapps.stackexchange.com/questions/45254/file-size-and-storage-limits-on-github#84746 and then what will you do when GitHub comes asking you to reduce the repository size?

https://git-lfs.github.com/[Git LFS] is one approach to deal with this, but we feel that it adds too much development overhead.

==== Store images in Wikimedia Commons

Wikimedia Commons is another great possibility to upload your images to:
\CirodownExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/Gel_electrophoresis_insert_comb.jpg/450px-Gel_electrophoresis_insert_comb.jpg]
{source=https://commons.wikimedia.org/wiki/File:Gel_electrophoresis_insert_comb.jpg}
]]

Cirodown likes Wikimedia Commons so much that we automatically parse the image URL and if it is from Wikimedia Commons, automatically deduce the `source` for you. So the above image renders the same without the `source` argument:
\CirodownExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
]]

And like for non-Wikimedia images, you can automatically generate a `title` from the `src` by setting the `title_from_src` \x[boolean-named-arguments][boolean named argument] or if `title-from-src` is set as the default \x[media-providers][media provider] for the media type: 
\CirodownExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
{title_from_src}
]]

And a quick test for a more complex thumb resized URL:
\CirodownExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/Gel_electrophoresis_insert_comb.jpg/450px-Gel_electrophoresis_insert_comb.jpg]
]]

If you really absolutely want to turn off the `source`, you can explicitly set:
\CirodownExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
{source=}
]]
but you don't want to do that for the most commonly Wikimedia Commons used license of CC-BY+ , do you? :-)

Upsides of using Wikimedia Commons for your images:
* makes it easier for other writers to find and reuse your images
* automatically generates resized versions of the uploaded images into several common dimensions so you can pick the smallest one that fits your desired \x[image-height] to reduce bandwidth usage
* if you have so many images that they would blow even the size of a \x[store-images-in-a-separate-media-repository][separate media repository], this will still work
Downsides:
* forces you to use the Creative Commons license
* requires the content to be educational in nature
* uploading a bunch of images to Wikimedia Commons does feel a bit more laborious than it should because you have to write down so much repeated metadata for them

=== Image lazy loading

We do this by default because Cirodown is meant to allow producing huge single page documents like Ciro likes it, and in this way:
* images that the user is looking at will load first
* we save a lot of bandwidth for the user who only wants to browse one section

TODO: maybe create a mechanism to disable this for the entire build with \x[cirodown-json].

=== Background color of transparent images

For the love of God, there is no standardized for SVG to set its background color without a rectangle? https://stackoverflow.com/questions/11293026/default-background-color-of-svg-root-element `viewport-fill` was just left in limbo?

And as a result, many many many SVG online images that you might want to reuse just rely on white pages and don't add that background rectangle.

Therefore for now we just force white background on \x[overview-of-files-in-this-repository][our default CSS], which is what most SVGs will work with. Otherwise, you can lose the entire image to our default black background.

Then if someone ever has an SVG that needs another background color, we can add an image attribute to set that color as a local style.

=== Image generators

TODO implement: mechanism where you enter a textual description of the image inside the code body, and it then converts to an image, adds to the `-media` repo and pushes all automatically. Start with dot.

https://github.com/cirosantilli/cirodown/issues/40

== Videos (`\Video` and `\video`)
{id=videos}

Very analogous to \x[images], only differences will be documented here.

In the case of videos, \x[where-to-store-images] becomes even more critical since videos are even larger than images, such that the following storage approaches are impractical off the bat:
* \x[store-images-inside-the-repository-itself]
* \x[store-images-in-a-separate-media-repository]
As a result, then https://commons.wikimedia.org[Wikimedia Commons] is one of the best options \x[store-images-in-wikimedia-commons][much like for images]:
\CirodownExample[[
\Video[https://upload.wikimedia.org/wikipedia/commons/8/85/Vacuum_pump_filter_cut_and_place_in_eppendorf.webm]
{id=sample-video-in-wikimedia-commons}
{title=Nice sample video stored in Wikimedia Commons.}
{start=5}
]]
We also handle more complex transcoded video URLs just fine:
\CirodownExample[[
\Video[https://upload.wikimedia.org/wikipedia/commons/transcoded/1/19/Scientific_Industries_Inc_Vortex-Genie_2_running.ogv/Scientific_Industries_Inc_Vortex-Genie_2_running.ogv.480p.vp9.webm]
{id=sample-video-in-wikimedia-commons-transcoded}
{title=Nice sample video stored in Wikimedia Commons transcoded.}
]]
Commons is better than YouTube if your content is on-topic there because:
* they have no ads
* it allows download of the videos: https://www.quora.com/Can-I-download-Creative-Commons-licensed-YouTube-videos-to-edit-them-and-use-them[].
* it makes it easier for other users to find and re-use your videos

If your video does not fit the above Wikimedia Commons requirements, YouTube could be a good bet. Cirodown https://github.com/cirosantilli/cirodown/issues/50[automatically detects YouTube URLs] for you, so the following should just work:
\CirodownExample[[
\Video[https://youtube.com/watch?v=YeFzeNAHEhU&t=38]
{id=sample-video-from-youtube-implicit-youtube}
{title=Nice sample video embedded from YouTube implicit from `youtube.com` URL.}
]]
The `youtu.be` domain hack URLs also work;
\CirodownExample[[
\Video[https://youtu.be/YeFzeNAHEhU?t=38]
{id=sample-video-from-youtube-implicit-youtu-be}
{title=Nice sample video embedded from YouTube implicit from `youtu.be` URL.}
]]
Alternatively, you can reach the same result in a more explicit and minimal way by setting `{provider=youtube}` and the `start` arguments:
\CirodownExample[[
\Video[YeFzeNAHEhU]{provider=youtube}
{id=sample-video-from-youtube-explicit}
{title=Nice sample video embedded from YouTube with explicit `youtube` argument.}
{start=38}
]]
When the `youtube` provider is selected, the Video address should only to contain the YouTube video ID, which shows in the YouTube URL for the video as:
``
https://www.youtube.com/watch?v=<video-id>
``
Remember that you can also enable the `youtube` provider by default on your \x[cirodown-json] with:
``
"media-provider" {
  "youtube": {"default-for": "video"}
}
``

But you can also use raw video files from any location that can serve them of course, e.g. here is one stored in this repository: \x[sample-video-in-repository].
\CirodownExample[[
\Video[Tank_man_side_hopping_in_front_of_some_tanks.mp4]
{id=sample-video-in-repository}
{title=Nice sample video stored in this repository.}
{source=https://www.youtube.com/watch?v=YeFzeNAHEhU}
{start=3}
]]

And as for images, setting `title_from_src` automatically calculates a title for you:
\CirodownExample[[
\Video[Tank_man_side_hopping_in_front_of_some_tanks.mp4]
{title_from_src}
{source=https://www.youtube.com/watch?v=YeFzeNAHEhU}
]]

=== Video `start` argument

The time to start playing the video at in seconds. Works for both `youtube` and non-Youtube videos.

=== Video lazy loading

Unlike \x[image-lazy-loading], we don't support video lazy loading yet because:
* non-`youtube` videos use the `video` tag which has no `loading` property yet
* `youtube` videos are embedded with `iframe` and `iframe` has no `loading` property yet

Both of this cases could be worked around with JavaScript:
* non-`youtube`: set `src` from JavaScript as shown for images: https://stackoverflow.com/questions/2321907/how-do-you-make-images-load-lazily-only-when-they-are-in-the-viewport/57389607#57389607[].

  But this breaks page semantics however, we don't know how to work around that
* `youtube` videos: same as above for the `iframe`, but this should be less problematic since YouTube videos are not viewable without JavaScript anyways, and who cares about `iframe` semantics?

== Code (\c[[`]], \c[[``]], `\c` and `\C`)
{id=code}

Inline code (code that should appear in the middle of a paragraph rather than on its own line) is done with a single backtick (\c[[`]]) \x[insane-macro-shortcuts][insane macro shortcut]:
\CirodownExample[[
My inline `x = 'hello\n'` is awesome.
]]
and block code (code that should appear on their own line) is done with two or more backticks (\c[[``]]):
\CirodownExample[[
``
f() {
  return 'hello\n';
}
``
]]

The sane version of inline code is a lower case `c`:
\CirodownExample[[[
My inline \c[[x = 'hello\n']] is awesome.
]]]
and the sane version of block math is with an upper case `C`:
\CirodownExample[[[
\C[[
f() {
  return 'hello\n';
}
]]
]]]

The capital vs lower case theme is also used in other elements, see: \x[block-vs-inline-macros].

If the content of the sane code block has many characters that you would need to \x[escape-characters][escape], you will often want to use \x[literal-arguments], which work just like the do for any other argument. For example:
\CirodownExample[[[[
\C[[[
A paragraph.

\C[[
And now, some long, long code, with lots
of chars that you would need to escape:
\ [  ] {  }
]]

A paragraph.
]]]
]]]]
Note that the initial newline is skipped automatically in code blocks, just as for any other element, due to: \x[argument-leading-newline-removal], so you don't have to worry about it.

The distinction between inline `\c` and block `\C` code blocks is needed because in HTML, https://stackoverflow.com/questions/5371787/can-i-have-a-pre-tag-inside-a-p-tag-in-tumblr/58603596#58603596[`pre` cannot go inside `P`].

We could have chosen to do some magic to differentiate between them, e.g. checking if the block is the only element in a paragraph, but we decided not to do that to keep the language saner.

And now a code block outside of \x[cirodown-example] to test how it looks directly under \x[toplevel]:

``
Hello
Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello
Hello
``

\Comment[[[[
TODO implement.
We can have cross references to code blocks as for other elements such as \x[images]:
\CirodownExample[[[
See this awesome code \x[my-code]:
``
ab
cd
``
{id=my-code}
]]]
]]]]

Block code can have a title, e.g. see this one: \x[code-my-nice-code]{full}:
\CirodownExample[[
``
ab
cd
``
{id=code-my-nice-code}
{title=My nice code block.}
]]

== Mathematics
{title2=`$`, `$$`, `\m` and `\M`}

Via https://katex.org/[KaTeX] server side, oh yes!

Inline math is done with the dollar sign (`$`) \x[insane-macro-shortcuts][insane macro shortcut]:
\CirodownExample[[
My inline $\sqrt{1 + 1}$ is awesome.
]]
and block math is done with two or more dollar signs (`$$`):
\CirodownExample[[
$$
\sqrt{1 + 1} \\
\sqrt{1 + 1}
$$
]]

The sane version of inline math is a lower case `m`:
\CirodownExample[[[
My inline \m[[\sqrt{1 + 1}]] is awesome.
]]]
and the sane version of block math is with an upper case `M`:
\CirodownExample[[[
\M[[
\sqrt{1 + 1} \\
\sqrt{1 + 1}
]]
]]]

The capital vs lower case theme is also used in other elements, see: \x[block-vs-inline-macros].

In the sane syntax, \x[escape-characters][as with any other argument], you have to either escape any closing square brackets `]` with a backslash `\`:
\CirodownExample[[
My inline \m[1 - \[1 + 1\] = -1] is awesome.
]]
or with the equivalent double open and close:
``
My inline \m[[1 - [1 + 1] = -1]] is awesome.
``

HTML escaping happens as you would expect, e.g. < shows fine in:
\CirodownExample[[
$$
1 < 2
$$
]]

Equation IDs and titles and linking to equations works identically to \x[images], see that section for full details. Here is one equation reference example that links to the following insane syntax equation: \x[eq-my-first-insane-equation]:
\CirodownExample[[
$$
\sqrt{1 + 1}
$$
{title=My first insane equation.}
]]
and the sane equivalent \x[eq-my-first-sane-equation]:
\CirodownExample[[[
\M{title=My first sane equation.}[[
\sqrt{1 + 1}
]]
]]]

Here is a raw one just to test the formatting outside of a `cirodown_comment`:
$$\sqrt{1 + 1}$$

=== Math defines across blocks

First here is an invisible block (with `{show=0}`) defining with a `\newcommand` definition after this paragraph:
\CirodownExample[[
$$
\newcommand{\foo}[0]{bar}
$${show=0}
]]
We make it invisible because this block only contains KaTeX definitions, and should not render to anything.

Then the second math block uses those definitions:
$$
\foo
$$

Analogously with `\def`, definition:
\CirodownExample[[
$$
\gdef\foogdef{bar}
$${show=0}
]]
and the second block using it:
\CirodownExample[[
$$
\foogdef
$$
]]

And just to test that `{show=1}` actually shows, although it is useless, and that `{show=0}` skips incrementing the equation count:
\CirodownExample[[
$$1 + 1$${show=1}
$$2 + 2$${show=0}
$$3 + 3$${show=1}
]]

== Tables
{title2=`|| `, `| `, `\Table`, `\Tr`, `\Th` and `\Td`}

The \x[insane-code-and-math-shortcuts][insane] syntax marks:
* headers with `|| ` (pipe, pipe space) at the start of a line
* regular cells with `| ` (pipe, space) at the start of a line
* separates rows with double newline
For example:
\CirodownExample[[
|| Header 1
|| Header 2

| 1 1
| 1 2

| 2 1
| 2 2
]]
Empty cells are allowed without the trailing space however:
\CirodownExample[[
| 1 1
|
| 1 3

| 2 1
|
| 2 3
]]

Equivalent fully explicit version:
\CirodownExample[[
\Table[
\Tr[
  \Th[Header 1]
  \Th[Header 2]
]
\Tr[
  \Td[1 1]
  \Td[1 2]
]
\Tr[
  \Td[2 1]
  \Td[2 2]
]
]
]]
Any white space indentation inside an explicit `\Tr` can make the code more readable, and is automatically removed from final output due to \x[remove-whitespace-children] which is set for `\Table`.

To pass further arguments to an implicit table such as `title` or `id`, you need to use an explicit `table` macro as in: \x[table-my-table].
\CirodownExample[[
\Table
{title=My table title.}
{id=table-my-table}
[
|| Header 1
|| Header 2

| 1 1
| 1 2

| 2 1
| 2 2
]
]]
The rules of when the caption shows up or not are the same as described for \x[images].

Multiple source lines, including paragraphs, can be added to a single cell with insane syntax by indenting the cell with exactly two spaces just as for \x[lists], e.g.:
\CirodownExample[[
|| h1
|| h2
|| h3

  h3 2

| 11
| 12

  12 2
| 13

| 21
| 22
| 23
]]
Arbitrarily complex nested constructs may be used, e.g. a table inside a list inside table:
\CirodownExample[[
| 00
| 01

  * l1
  * l2

    | 20
    | 21

    | 30
    | 31

| 10
| 11
]]

And now a table outside of \x[cirodown-example] to test how it looks directly under \x[toplevel]:

\Table{title=My table title.}
[
\Tr[
  \Th[Header 1]
  \Th[Header 2]
]
\Tr[
  \Td[1 1]
  \Td[1 2]
]
\Tr[
  \Td[2 1]
  \Td[2 2]
]
]

And a fully insane one:

|| Header 1
|| Header 2

| 1 1
| 1 2

| 2 1
| 2 2

=== Table sorting

JavaScript interactive on-click table sorting is enabled by default, try it out by clicking on the header row:
\CirodownExample[[
|| String col
|| Integer col
|| Float col

| ab
| 2
| 10.1

| a
| 10
| 10.2

| c
| 2
| 3.4
]]
Powered by: https://github.com/tristen/tablesort

== Quotations
{title2=`\Q`}

With `q`:
\CirodownExample[[
And so he said:

\Q[
Something very smart

And with multiple paragraphs.
]

and it was great.
]]

== Bold
{title2=`\b`}

\CirodownExample[[
Some \b[bold] text.
]]

== Italic
{title2=`\i`}

\CirodownExample[[
Some \i[italic] text.
]]

== Passthrough
{title2=`\Passthrough`}

Dumps its contents directly into the rendered output.

This construct is not XSS safe, see: \x[xss-unsafe]{full}.

Here for example we define a paragraph in raw HTML:
\CirodownExample[[[
\Passthrough[[
<p>Hello <b>raw</b> HTML!</p>
]]
]]]

== Comments

The `Comment` and `comment` macros are regular macros that does not produce any output. Capitalization is explained at: \x[block-vs-inline-macros]{full}.

You will therefore mostly want to use it with a \x[literal-arguments][literal argument], which will, as for any other macro, ignore any macros inside of it.
\CirodownExample[[[
Before comment.

\Comment[[
Inside comment.
]]

After comment.
]]]

And an inline one:
\CirodownExample[[[
My inline \comment[[inside comment]] is awesome.

\comment[[inside comment]] inline at the start.
]]]

== `JsCanvasDemo`
{id=jscanvasdemo}

The `JsCanvasDemo` macro allows you to create interactive HTML/JavaScript https://en.wikipedia.org/wiki/Canvas_element[canvas] demos easily.

These demos:
* only start running when the user scrolls over them for the first time
* stop automatically when they leave the viewport
so you can stuff as many of them as you want on a page, and they won't cause the reader's CPU to fry an egg.

\CirodownExample[[[
\JsCanvasDemo[[
new class extends CirodownCanvasDemo {
  init() {
    super.init('hello');
    this.pixel_size_input = this.addInputAfterEnable(
      'Pixel size',
      {
        'min': 1,
        'type': 'number',
        'value': 1,
      }
    );
  }
  draw() {
    var pixel_size = parseInt(this.pixel_size_input.value);
    for (var x = 0; x < this.width; x += pixel_size) {
      for (var y = 0; y < this.height; y += pixel_size) {
        var b = ((1.0 + Math.sin(this.time * Math.PI / 16)) / 2.0);
        this.ctx.fillStyle =
          'rgba(' +
          (x / this.width) * 255 + ',' +
          (y / this.height) * 255 + ',' +
          b * 255 +
          ',255)'
        ;
        this.ctx.fillRect(x, y, pixel_size, pixel_size);
      }
    }
  }
}
]]
]]]

And another one showing off some https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API[WebGL]:

\JsCanvasDemo[[
new class extends CirodownCanvasDemo {
  init() {
    super.init('webgl', {context_type: 'webgl'});
    this.ctx.viewport(0, 0, this.ctx.drawingBufferWidth, this.ctx.drawingBufferHeight);
    this.ctx.clearColor(0.0, 0.0, 0.0, 1.0);
    this.vertexShaderSource = `
#version 100
precision highp float;
attribute float position;
void main() {
  gl_Position = vec4(position, 0.0, 0.0, 1.0);
  gl_PointSize = 64.0;
}
`;

    this.fragmentShaderSource = `
#version 100
precision mediump float;
void main() {
  gl_FragColor = vec4(0.18, 0.0, 0.34, 1.0);
}
`;
    this.vertexShader = this.ctx.createShader(this.ctx.VERTEX_SHADER);
    this.ctx.shaderSource(this.vertexShader, this.vertexShaderSource);
    this.ctx.compileShader(this.vertexShader);
    this.fragmentShader = this.ctx.createShader(this.ctx.FRAGMENT_SHADER);
    this.ctx.shaderSource(this.fragmentShader, this.fragmentShaderSource);
    this.ctx.compileShader(this.fragmentShader);
    this.program = this.ctx.createProgram();
    this.ctx.attachShader(this.program, this.vertexShader);
    this.ctx.attachShader(this.program, this.fragmentShader);
    this.ctx.linkProgram(this.program);
    this.ctx.detachShader(this.program, this.vertexShader);
    this.ctx.detachShader(this.program, this.fragmentShader);
    this.ctx.deleteShader(this.vertexShader);
    this.ctx.deleteShader(this.fragmentShader);
    if (!this.ctx.getProgramParameter(this.program, this.ctx.LINK_STATUS)) {
      console.log('error ' + this.ctx.getProgramInfoLog(this.program));
      return;
    }
    this.ctx.enableVertexAttribArray(0);
    var buffer = this.ctx.createBuffer();
    this.ctx.bindBuffer(this.ctx.ARRAY_BUFFER, buffer);
    this.ctx.vertexAttribPointer(0, 1, this.ctx.FLOAT, false, 0, 0);
    this.ctx.useProgram(this.program);
  }
  draw() {
    this.ctx.clear(this.ctx.COLOR_BUFFER_BIT);
    this.ctx.bufferData(this.ctx.ARRAY_BUFFER, new Float32Array([Math.sin(this.time / 60.0)]), this.ctx.STATIC_DRAW);
    this.ctx.drawArrays(this.ctx.POINTS, 0, 1);
  }
}
]]

== `CirodownExample`
{id=cirodown-example}

Shows both the Cirodown code and its rendered output, e.g.:
\CirodownExample[[[
\CirodownExample[[
Some `ineline` code.
]]
]]]

Its input should be thought of as a literal code string, and it then injects the rendered output in the document.

This macro is used extensively in the Cirodown documentation.

== Cirodown syntax

=== Insane macro shortcuts

Certain commonly used macros have insane macro shortcuts that do not start with backslash (`\`):
* \x[paragraphs]: `\n\n` (double newline)
* \x[links]: `a http://exmaple.com b` (space followed by `http://`)
* \x[mathematics]: `$`, described at: \x[insane-code-and-math-shortcuts]
* \x[code]: \c[[`]], described at: \x[insane-code-and-math-shortcuts]
* \x[lists]: `* ` and `  ` indentation, described at: \x[lists]

Originally, \x[design-goals][Ciro wanted to avoid those], but they just feel too good to avoid.

Every insane syntax does however have an equivalent sane syntax.

The style recommendation is: use the insane version which is shorter, unless you have a specific reason to use the sane version.

==== Insane code and math shortcuts

The insane code and math shortcuts work very analogously and are therefore described together in this section.

The insane inline code syntax:
\CirodownExample[[
a `b c` d
]]
and is equivalent to the sane:
``
a \c[[b c]] d
``

The insane block code:
\CirodownExample[[
a

``
b c
``

d
]]
and is equivalent to the sane:
``
a

\C[[
b c
]]

d
``

==== Insane macro shortcut extra arguments

Insane arguments always work by abbreviating:
* the macro name
* one or more of its positional arguments, which are fixed as either \x[literal-arguments][literal or non-literal] for a given insane construct
This means that you can add further arguments as usual.

For example, an insane code block with an id can be written as:
``
a `b c`{id=ef} g
``
because that is the same as:
\CirodownExample[[
a \c[b c]{id=ef} g
]]
So we see that the `b c` argument is the very first argument of `\c`.

Extra arguments must come after the insane opening, e.g. the following does not work:
\CirodownExample[[
a {id=ef}`b c` g
]]

This restriction things easy to parse for humans and machines alike.

==== Escapes in insane macro shortcuts

Literal backticks and dollar signs can be produced witha backslash escape as in:
\CirodownExample[[
a \` \$ b
]]

It is not possible to escape backticks (\c[[`]]) inside an insane inline code, or dollar signs (`$`) in insane math.

The design reason for that is because multiple backticks produce block code.

The upside is that then you don't have to escape anything else, e.g. backslashes (`\`) are rendered literally.

The only way to do it is to use the sane syntax instead:
\CirodownExample[[[
a \c[[b ` c]] d

a \m[[\sqrt{\$4}]] d
]]]

Within block code and math, you can just add more separators:
\CirodownExample[[
```
code with two backticks
``
nice
```
]]

=== Macro argument syntax

==== Positional vs named arguments (`[...]` vs `{key=...}`)
{id=positional-vs-named-arguments}

Every argument in Cirodown is either positional or named.

For example, in a \x[headers]{p=0} definition with an ID:
``
= My asdf{id=asdf qwer}{scope}
``
we have:
* two positional argument: `[1]` and `[My asdf]`. Those are surrounded by square brackets `[]` and have no name
* two named arguments: `{id=asdf qwer}` and `{scope}`.

  The first one has name `id` and the mandatory separator `=`, followed by the value `asdf qwer`.

  The `scope` one does not need `=` because it is a \x[boolean-named-arguments].

You can determine if a macro is positional or named by using \x[help-macros]. Its output contains something like:
``
  "h": {
    "name": "h",
    "positional_args": [
      {
        "name": "level"
      },
      {
        "name": "content"
      }
    ],
    "named_args": {
      "id": {
        "name": "id"
      }
      "scope": {
        "name": "scope"
      }
    },
``
and so we see that `level` and `content` are positional arguments, and `id` and `scope` are named arguments.

Generally, positional arguments are few (otherwise it would be hard to know which is which is which), and are almost always used for a given element so tha they save us from typing the name too many times.

The order of positional arguments must of course be fixed, but named arguments can go anywhere. We can even mix positional and named arguments however we want, although this is not advised for clarity.

The following are therefore all equivalent:
``
\H[1][My asdf]{id=asdf qwer}{scope}
\H[1][My asdf]{scope}{id=asdf qwer}
\H{id=asdf qwer}{scope}[1][My asdf]
\H{scope}[1]{id=asdf qwer}[My asdf]
``

Just like named arguments, positional arguments are never mandatory.

===== Positional argument default values

Most positional arguments will default to an empty string if not given.

However, some positional arguments can have special effects if not given.

For example, an anchor with the first positional argument present (the URL), but not the second positional argument (the link text) as in:
\CirodownExample[[
\a[http://example.com]
]]
has the special effect of generating automatic links as in:
``
\a[http://example.com][http://example.com]
``

This can be contrasted with named aguments, for which there is always a default value, notably for \x[boolean-named-arguments].

See also: \x[links].

====== mandatory positional arguments

Some positional arguments are required, and if not given cirodown reports an error and does not render the node.

This is for example the `level` of a \x[headers]{p=0}.

These arguments marked with the `mandatory: true` \x[help-macros] argument property.

===== Boolean named arguments

Name arguments marked in \x[help-macros] as `boolean: true` must either:
* take no value and no `=` sign, in which case the value is implicitly set to `1`
* take value exactly `0` or `1`
* not be given, in which case the default value is the `default` from \x[help-macros], or `0` if such default is not given

For example, \x[cross-reference-full-argument][the `full` argument] of \x[internal-cross-references] is correctly written as:
\CirodownExample[[
\x[boolean-named-arguments]{full}
]]
without the `=` sign, or equivalently:
\CirodownExample[[
\x[boolean-named-arguments]{full=1}
]]
The `full=0` version is useful in the case of reference targets that unlike \x[headers] expand the title on the cross reference by default, e.g. \x[images]:
\CirodownExample[[
\x[boolean-named-arguments]{full=1}
]]

The name "boolean argument" is given by analogy to the https://stackoverflow.com/questions/16109358/what-is-the-correct-readonly-attribute-syntax-for-input-text-elements/24588427#24588427["boolean attribute" concept in HTML5].

==== JavaScript interface for arguments

The JavaScript interface sees arguments as follows:
``
function macro_name(args)
``
where args is a dict such that:
* optional arguments have the key/value pairs explicitly given on the call
* mandatory arguments have a key documented by the API, and the value on the call.

  For example, the link API names its arguments `href` and `text`.

==== Literal arguments (`[[...]]` and `{{key=...}}`)
{id=literal-arguments}

Arguments that are opened with more than one square brackets `[` or curly braces `{` are literal arguments.

In literal arguments, Cirodown is not parsed, and the entire argument is considered as text until a corresponding close with the same number of characters.

Therefore, you cannot have nested content, but it makes it extremely convenient to write \x[code] or \x[mathematics].

For example, a multiline code block with double open and double close square brackets inside can be enclosed in triple square brackets:
\CirodownExample[[[
A literal argument looks like this in Cirodown:

\C[[
\C[
A multiline

code block.
]
]]

And another paragraph.
]]]

The same works for inline code:
\CirodownExample[[[
The program \c[[puts("]");]] is very complex.
]]]

Within literal blocks, only one thing can be escaped with backslashes are:
* leading open square bracket `[`
* trailing close square bracket `]`

The rule is that:
* if the first character of a literal argument is a sequence of backslashes (`\`), and it is followed by another argument open character (e.g. `[`, remove the first `\` and treat the other characters as regular text
* if the last character of a literal argument is a `\`, ignore it and treat the following closing character (e.g. `]`) as regular text

See the following open input/output pairs:
``
\c[[\ b]]
<code>\ b</code>

\c[[\a b]]
<code>\a b</code>

\c[[\[ b]]
<code>[ b</code>

\c[[\\[ b]]
<code>\[ b</code>

\c[[\\\[ b]]
<code>\\[ b</code>
``
and close examples:
``
\c[[a \]]
<code>a \</code>

\c[[a \]]]
<code>a ]</code>

\c[[a \\]]]
<code>a \]</code>
``

==== Argument leading newline removal

If the very first character of an argument is a newline, then that character is ignored if it would be part of a regular plaintext node.

For example:
``
\C[[
a

b
]]
``
generates something like:
``
<pre><code>a

b
</code></pre>
``
instead of:
``
<pre><code>
a

b
</code></pre>
``
This is extremely convenient to improve the readability of code blocks and similar constructs.

The newline is however considered if it would be part of some \x[insane-macro-shortcuts]{p=0}. For example, we can start an \x[lists][insane list] inside a \x[quotations]{p=0} as in:
\CirodownExample[[
\Q[
* a
* b
]
]]
where the insane list requires a leading newline `\n* ` to work. That newline is not ignored, even though it comes immediately after the `\Q[` opening.

==== Argument newlines between arguments removal

The macro name and the first argument, and two consecutive arguments, can be optionally separated by exactly one newline character, e.g.:
``
\H[2]
{scope}
[Design goals]
``
is equivalent to:
``
\H[2]{scope}[Design goals]
``
and this non-recommended mixed style:
``
\H[2]{scope}
[Design goals]
``
This allows to greatly improve the readability of long argument lists by having them one per line.

There is one exception to this however: inside an \x[headers][insane header], any newline is interpreted as the end of the insane header. This is why the following works as expected:
``
== My header 2 `some code`
{id=asdf}
``
and the `id` gets assigned to the header rather than the trailing code element.

==== Escape characters

For \x[literal-arguments][non-literal macro arguments], you have to use a backslash to escape:
* `\`: backslashes start macros
* `\` and `\`: open and close \x[positional-vs-named-arguments][positional macro arguments]
* `\` and `\`: open and close \x[positional-vs-named-arguments][optional macro arguments]
* `$` (dollar sign): \x[insane-macro-shortcuts][insane macro shortcut] for \x[mathematics]
* \c[[`]] (backtick): \x[insane-macro-shortcuts][insane macro shortcut] for \x[code]

The escape rules for literal arguments are described at: \x[literal-arguments]{full}.

This is good for short arguments of regular text, but for longer blocks of \x[code] or \x[mathematics], you may want to use \x[literal-arguments]

==== `auto_parent` macro property
{id=auto-parent}

Some sequences of macros such as `l` from \x[lists] and `tr` from \x[tables] automatically generate implicit parents, e.g.:
``
\Ul[
\L[aa]
\L[bb]
]
``
parses exactly like:
``
\L[aa]
\L[bb]
``

The children are always added as arguments of the `content` argument of the implicit parent.

If present, the `auto_parent` macro property determines which auto-parent gets added to those macros.

==== `remove_whitespace_children` macro argument property
{id=remove-whitespace-children}

In HTML, certain elements such as `<ul>` cannot have any `text` nodes in them, and any whitespace is ignored, see https://stackoverflow.com/questions/2161337/can-we-use-any-other-tag-inside-ul-along-with-li/60885802#60885802[].

A similar concept applies to Cirodown, e.g.:
``
\Ul[
\L[aa]
\L[bb]
]
``
does not parse as:
``
\Ul[\L[aa]<NEWLINE>\L[bb]<NEWLINE>]
``
but rather as:
``
\Ul[\L[aa]\L[bb]]
``
because the `content` argument of `ul` is marked with `remove_whitespace_children` and automatically removes any whitespace children (such as a newline) as a result.

This also applies to consecutive sequences of \x[auto-parent] macros, e.g.:
``
\L[aa]
\L[bb]
``
also does not include the newline between the list items.

The definition of whitespace is the same as the ASCII whitespace definition of HTML5: ` \r\n\f\t`.

=== Block vs inline macros

Every Cirodown macro is either block or inline:
* a block macro is one that takes up the entire line when rendered

  All block macros start with a capital letter, e.g. `\H` for \x[headers].
* and an inline macro is one that goes inside of a line.

  Every inline macro starts with a lowercase letter e.g. `\a` for \x[links].

Some macros have both a block and an inline version, and like any other macro, those are differentiated by capitalization:
* \x[mathematics]
* \x[code]
* \x[comments]

=== Plaintext conversion

In some cases, we want to extract plaintext versions of complex nested elements.

TODO implement:
``
For example, when the \x[internal-cross-references][ID of a node is derived from its title], we don't want the default element to have complex HTML tags on it:
\CirodownExample[[
\x[title-with-code]

\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=title `with code`.}
]]
In this example, see how the contents of the code block \c[`with code`] were rendered as plaintext and used for the ID.
``

=== Known URL protocols

Certain common URL protocols are treated as "known" by Cirodown, and when found they have special effects in some parts of the conversion.

The currently known protocols are:
* `http://`
* `https://`

Effects of known protocols include:
* \x[insane-link-parsing-rules]: mark the start of insane links
* \x[store-images-in-a-separate-media-repository]: mark an image `src` to ignore `provider`

=== Security

Cirodown HTML output is designed to be XSS safe by default, any non-XSS safe constructs must be enabled with a non-default flag or setting, see: \x[xss-unsafe].

Of course, we are walking on eggs, and this is hard to assert, so the best thing to do later on will be to parse the output e.g. with https://developer.mozilla.org/en-US/docs/Web/API/DOMParser[`DOMParser`] to ensure that it is valid and does not contain any `script` tags, but it is not as simple as that: https://stackoverflow.com/questions/37435077/execute-javascript-for-xss-without-script-tags/61588322#61588322

About server safety, we have to think about it. E.g. stuff like \x[prepublish] already allows ACE.

==== xss-unsafe
{title2=`--xss-unsafe`}

XSS unsafe constructs lead to errors by default. XSS unsafe constructs can be allowed \x[cirodown-executable][from the command] line with:
``
./cirodown --xss-unsafe
``
or from the \x[cirodown-json] file with an entry of form:
``
"xss-unsafe": true
``

== Tooling

Unlike all languages which rely on ad-hoc tooling, we will support every single tool that is required and feasible to be in this repository in this repository, in a centralized manner.

=== `cirodown` executable
{id=cirodown-executable}

Convert a `.ciro` file to HTML and output the HTML to a file with the same basename without extension, e.g.:
``
cirodown hello.ciro
firefox hello.html
``

Files named `README.ciro` are automatically converted to `index.html` so that they will show on both GitHub READMEs and at the website's base address:
``
cirodown README.ciro
firefox hello.html
``

The output file can be selected explicitly with: \x[outfile].

Output to stdout instead of saving it to a file:
``
cirodown --stdout README.ciro
``

Convert all `.ciro` files in a directory to HTML files next to each corresponding `.ciro` file, e.g. `somefile.ciro` to `somefile.html`:
``
cirodown .
``

In order to resolve \x[internal-cross-file-references], this actually does two passes:
* first an ID extraction pass, which parses all inputs and dumps their IDs to the ID database
* then a second render pass, which uses the IDs in the ID database

Convert a `.ciro` file from stdin to HTML and output the contents of `<body>` to stdout:
``
printf 'ab\ncd\n' | cirodown --body-only
``

==== Index files

The following basenames are considered "index files":
* `README.ciro`
* `index.ciro`

Those basenames have the following magic properties:
* the default output file name for an index file in HTML output is always `index.html`, including that of `README.ciro`. This way it will appear on both the root of the HTML output and on the GitHub home page once GitHub adds Cirodown support.
* the default \x[the-toplevel-header][toplevel header] ID of an index files is derived from the parent directory basename rather than from the source file basename

==== `cirodown` executable options
{id=cirodown-executable-options}

===== `--dry-run`
{id=dry-run}

The `--dry-run` option is a good way to debug \x[publish][`--publish` option], as it builds the publish output files without doing any git commands that would be annoying to revert. So after doing:
``
./cirodown --dry-run --publish .
``
you can just go and inspect the generated HTML to see what would get pushed at:
``
cd out/publish/out/publish/
``

===== `--generate`
{id=generate}

Generate a simple repository template for a new project.

This includes common files that almost all projects will want to have, e.g.:
* `package.json` specifying the Cirodown version
* `.gitignore` to prevent output from being committed accidentally

The output provides a hello world setup that is immediately ready to be published e.g. \x[publish-to-github-pages][to GitHub pages].

===== `--generate-multifile`
{id=generate-multifile}

Like \x[generate][`--generate`], but generate a more complex template that serves as a good starting point for a multifile website.

===== `--help-macros`
{id=help-macros}

You can get an overview of all macros in JSON format with:
``
cirodown --help-macros
``

===== `--html-embed`
{id=html-embed}

Embed as many external resources as possible into a single HTML file.

The use case for this option is to produce a single HTML file for an entire build that is fully self contained, and can therefore be given to consumers and viewed offline, much like a PDF.

Examples of embeddings done:
* CSS and JavaScript are copy pasted in place into the HTML.

  The default built-in CSS and JavaScript files used by Cirodown (e.g. the KaTeX CSS \x[mathematics][used for mathematics]) are currently all automatically downloaded as NPM package dependencies to cirodown

  Without `--html-embed`, those CSS and JavaScript use their main cloud CDN URLs, and therefore require Internet connection to view the generated documents.

  The embedded version of the document can be viewed offline however.
* \x[images] are downloaded if needed and embedded as `data:` URLs.

  Images that are managed by the project itself and already locally present, such as those inside the project itself or due to \x[media-providers] usually don't require download.

  For images linked directly from the web, we maintain a local download cache, and skip downloads if the image is already in the cache.

  To re-download due to image updates, use either:

  * `--asset-cache-update`: download all images such that the local disk timestamp is older than the HTTP modification date with https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since[`If-Modified-Since`]
  * `--asset-cache-update-force`: forcefully redownload all assets

Keep in mind that certain things can never be embedded, e.g.:
* YouTube videos, since YouTube does not offer any download API

===== `--html-single-page`
{id=html-single-page}

If given:
* the \x[includes][`include`] macro parses the target file to be included and includes it in-place
* \x[internal-cross-file-references] are disabled, and the cross file ID database does not get updated.

  It should be possible to work around this, but we are starting with the simplest implementation that forbids it.

  The problem those cause is that the IDs of included headers show as duplicate IDs of those in the ID database.

  This should be OK to start with because the more common use case with `--html-sinle-page` is that of including all headers in a single document.

Otherwise, `include` only adds the headers of the other file to the table of contents of the current one, but not the body of the other file. The ToC entries then point to the headers of the included external files.

You may want to use this option together with \x[html-embed] to produce fully self-contained individual HTML files for your project.

===== `--no-html-x-extension`
{id=html-x-no-extension}

If not given, \x[internal-cross-references] render with the `.html` extension as in:
``
<a href=not-readme.html#h2-in-not-the-readme>
``

This way, those links will work when rendering locally to `.html` files which is the default behaviour of:
``
cirodown .
``

If given however, the links render without the `.html` as in:
``
<a href=not-readme#h2-in-not-the-readme>
``
which is what is needed for servers such as GitHub Pages, which automatically remove the `.html` extension from paths.

This option is automatically implied when publishing to targets that remove the `.html` extension such as GitHub pages.

===== `--outfile <outfie>`
{id=outfile}

Save the output to a given file instead of outputting to stdout:
``
./cirodown --outfile not-readme.html not-readme.ciro
``

The generated output is slightly different than that of:
``
./cirodown not-readme.ciro > not-readme.html
``
because with `--outfile` we know where the output is going, and so we can generate relative includes to default CSS/JavaScript files.

===== `--publish`
{id=publish}

Cirodown tooling is so amazing that we also take care of the HTML publishing for you!

Once publish target is properly setup, all you have to do is run:
``
git add README.ciro
git commit -m 'more content!'
cirodown --publish
``
and your changes will be published to the  default target specified in \x[cirodown-json].

If not specified, the default target is \x[publish-to-github-pages].

Only changes committed to Git are pushed.

Files that `cirodown` knows how to process get processed and only their outputs are added to the published repo, those file types are:
* `.ciro` files are converted to `.html`
* `.scss` files are converted to `.css`
Every other Git-tracked file is pushed as is.

When `--publish` is given, stdin input is not accepted, and so the current directory is built by default, i.e. the following two are equivalent:
``
./cirodown --publish
./cirodown --publish .
``

Publishing only happens if the build has no errors.

====== Publish to GitHub Pages

https://pages.github.com/[GitHub pages] is the default Cirodown publish target.

The first step is of course to create the GitHub repository that will contain your GitHub pages as usual.

Then, if you want to publish to the GitHub pages root page, you need to setup the branches correctly as shown at: \x[publish-to-github-pages-root-page].

Finally, for both root and non-root pages, follow the steps at: \x[publish-to-github-common-steps].

======= Publish to GitHub pages root page

If you want to publish your root user page, which appears at `/` (e.g. https://github.com/cirosantilli/cirosantilli.github.io for the user `cirosantilli`), GitHub annoyingly forces you to use the `master` branch for the HTML output:
* https://github.com/isaacs/github/issues/212
* https://stackoverflow.com/questions/31439951/how-can-i-use-a-branch-other-than-master-for-user-github-pages

This means that you must place your `.ciro` input files in a branch other than `master` to clear up `master` for the generated HTML.

`cirodown` automatically detects if your repository ois a root repository or not by parsing `git remote` output, but you must setup the branches correctly yourself.

So on a new repository, you must https://stackoverflow.com/questions/42871542/how-to-create-a-git-repository-with-the-default-branch-name-other-than-master[first checkout to a different branch] as in:
``
git init
git checkout -b dev
``
or to move an existing repository to a non-master branch:
``
git checkout -b dev
git push origin dev:dev
git branch -D master
git push --delete origin master
``

You then will also want to set your default repository branch to `dev` in the settings for that repository: https://help.github.com/en/github/administering-a-repository/setting-the-default-branch

======= Publish to GitHub common steps

For non-root GitHub pages, no special setup is needed, `cirodown` automatically populates the `gh-pages` branch for you.

Once the repository branch is setup, we can just \x[generate][generate the repository template with `--generate`], make a commit, and immediately publish!
``
cirodown --generate
git add .
git commit --message 'initial commit from template'
cirodown --publish .
``

===== `--publish-commit <commit-message>`
{id=publish-commit}

Like \x[publish], but also automatically:
* `git add -u` to automatically add change to any files that have been previously git tracked
* `git commit -m <commit-message>` to create a new commit with those changes

This allows you to publish your changes live in a single command such as:
``
cirodown --publish-commit 'my amazing change' .
``

With great power comes great responsibility of course, but who cares!

===== `--show-headers`
{id=show-headers}

This nifty little option outputs to stderr what the header graph looks like!

It is a bit like a \x[table-of-contents] in your terminal, for when you need to have a look at the outline of the document to decide where to place a new header, but are not in the mood to open a browser or use the \x[editor-with-preview].

Sample output excerpt for this document:
``
= h1  cirodown
== h2 1 quick-start
== h2 2 design-goals
=== h3 2.1 saner
=== h3 2.2 more-powerful
== h2 3 paragraphs
== h2 4 links
``

This option can also serve as a debug tool for header tree related features (confession: that was its original motivation!).

===== `--watch`
{id=watch}

Don't quit `cirodown` immediately.

Instead, watch the selected file or directory for changes, and rebuild individual files when changes are detected.

Watch every `.ciro` file in an entire directory:
``
cirodown --watch .
``
When a directory is given as the input path, this automatically first does an ID extraction pass on all files to support \x[internal-cross-file-references].

Now you can just edit any Cirodown file such has `README.ciro`, save the file in your editor, and refresh the webpage and your change should be visible, no need to run a `cirodown` command explicitly every time.

Exit by entering Ctrl + C on the terminal.

Watch a single file:
``
cirodown --watch README.ciro
``
When a single file is watched, the reference database is not automatically updated. If it is not already up-to-date, you should first update it with:
``
cirodown .
``
otherwise you will just get a bunch of undefined ID errors every time the input file is saved.

TODO: integrate Live Preview: https://asciidoctor.org/docs/editing-asciidoc-with-live-preview/ to also dispense the browser refresh.

===== `--template`
{id=template}

Select a custom https://github.com/harttle/liquidjs[Liquid template] file for the output.

The recommended file name for this file is `main.liquid.html`, which is already ignored by default from the published output.

If not given, the default template at one point was:
``
<!doctype html>
<html lang=en>
<head>
<meta charset=utf-8>
<title>{{ title }}</title>
<style>{{ style }}</style>
</head>
<body class="cirodown">
{{ body }}
</body>
</html>
``
This will get out of sync sooner or later with the code, but this should still serve as a good base example for this documentation.

Defined variables:
* `body`: the rendered body
* `git_sha`: SHA of the latest git commit of the source code if in a git repository
* `style`: default Cirodown stylesheets
* `title`

We pick Liquid because it is server-side safe: if we ever some day offer a compilation service, Liquid is designed to prevent arbitrary code execution and infinite loops in templates.

==== `cirodown.json`
{id=cirodown-json}

Cirodown configuration file that affects the behaviour of cirodown for all files in the directory.

`cirodown.json` not used for input from stdin, since we are mostly doing quick tests in that case.

===== `media-providers`
{id=media-providers}

The `media-providers` entry of `cirodown.json` specifies properties of how media such as \x[images] and \x[videos][videos] are retrieved and rendered.

The general format of `media-providers` looks like:
``
"media-providers": {
  "github": {
    "default-for": ["image"], // "all" to default for both image, video and anything else
    "path": "data/media/",    // data is gitignored, but should not be nuked like out/
    "remote": "cirosantilli/cirodown-media",
  },
  "local": {
    "default-for": ["video"],
    "path": "media/",
  },
  "youtube": {}
}
``

Direct children of media-providers and subproperties that are valid only for them specifically:
* `local`: tracked in the current Git repository as mentioned at \x[store-images-inside-the-repository-itself]{full}
  * `path`: location of the cloned local repository relative to the root the main repository
* `github`: tracked in a separate Git repository as mentioned at \x[store-images-in-a-separate-media-repository]{full}
  * `path`: same as for `local`
  * `remote`: `<github-username>/<repo-name>`
* `youtube`: YouTube \x[videos][videos]

Properties that are valid for every provider:
* `default-for`: use this provider as the default for the given types of listed macros.

  The first character of the macros are case insensitive and must be given as lower case. Therefore e.g.:
  * `image` applies to both `image` and `Image`
  * giving `Image` is an error because that starts with an upper case character

See also: https://github.com/cirosantilli/cirodown/issues/40

===== `prepublish`
{id=prepublish}

Path of a script that gets executed before running \x[publish][`cirodown --publish`].

The script arguments are:
* the publish output directory.

  That directory is guaranteed to exist when `prepublish` is called.

  For `git`-based publish targets, all files are almost ready in there, just waiting for a `git add .` that follows `prepublish`.

  This means that you can use this script to place or remove files from the final publish output.

If the `prepublish` script returns with a non-zero exit value, the publish is aborted.

===== `redirects`
{id=redirects}

Create HTML redirects from \x[the-toplevel-header][toplevel headers] to arbitrary IDs, for example:
``
"redirects": {
  "a-toplevel-directory": "not-readme",
  "quick-start-redirect-test": "quick-start"
},
``
The HTML files are generated whenever a directory compilation is required, e.g. as in `cirodown .`.

The sources must be toplevel headers for now, because it is harder to implement redirections from `#someid` as that would require JavaScript: https://github.com/cirosantilli/cirodown/issues/48[].

This feature can be seen live on this document by visiting our tests:
* \a[not-readme-redirect-test]: redirects to \x[not-readme]
* \a[quick-start-redirect-test]: redirects to \x[quick-start]

To quickly test bugs in redirect generation, you may use the `--generate-redirects` flag as:
``
cirodown --generate-redirects .
``
which generates the redirects while skipping everything else.

=== Editor with preview

TODO

We must achieve an editor setup with synchronized live side-by-side preview.

Likely, we will first do a non WYSIWYG editor with side by side preview with scroll sync.

Then, if the project picks up steam, we can start considering a full WYSISYG.

It would be amazing to have a WebKit interface that works both on browser for the and locally.

Possibilities we could reuse:
* Editor.js

  Returns JSON AST!
  * website: https://editorjs.io/ json output
  * source: https://github.com/codex-team/editor.js
  * WYSIWYG: no
  * preview scroll sync: yes
* StackEdit
  * website: https://stackedit.io
  * source: https://github.com/benweet/stackedit
  * demo: https://stackedit.io/app
  * WYSIWYG: no
  * preview scroll sync: yes
* Editor.md
  * website: https://github.com/pandao/editor.md
  * source: https://github.com/pandao/editor.md
  * demo: https://pandao.github.io/editor.md
  * WYSIWYG: no
  * preview scroll sync: yes but buggy when tested 2019-12-12 on live website
* Quill.md
  * website: https://quilljs.com
  * source: https://github.com/pandao/editor.md
  * demo: https://pandao.github.io/editor.md
  * WYSIWYG: yes
  * markdown output: no https://github.com/quilljs/quill/issues/74
* https://ui.toast.com/tui-editor/
* https://www.froala.com/wysiwyg-editor

=== Conversion to/from other formats

The only thing we have for now is the quick and dirty \a[adoc-to-ciro].

The better approach would be to implement a converter in Haskell from anything to Cirodown.

And from Cirodown to anything, create new output formats inside Cirodown to those other formats.

== Developing Cirodown

=== Test system

Run all tests:
``
npm test
``

List all tests:
``
node node_modules/mocha-list-tests/mocha-list-tests.js main.js
``
as per: https://stackoverflow.com/questions/41380137/list-all-mocha-tests-without-executing-them/58573986#58573986[].

Run just one test by name:
``
npm test -- -g 'one paragraph'
``
as per: https://stackoverflow.com/questions/10832031/how-to-run-a-single-test-with-mocha TODO: what if the test name is a substring?

Step debug during a test run. Add the statement:
``
debugger;
``
to where you want to break in the code, and then:
``
node inspect ./node_modules/.bin/mocha test --ignore-leaks -g 'p with id before'
``

=== Overview of files in this repository

Source files:
* \a[index.js]: main code. Must be able to run in the browser, so no Node.js specifics. Exposes the central `convert` function
* \a[cirodown]: CLI executable. Is basically just a CLI interface frontend to `convert`
* \a[test.js]: contains all the Mocha tests, see also: \x[test-system]
* \a[README.md]: minimal Markdown README until GitHub / NPM support Cirodown :-)
* \a[cirodown.runtime.js]: JavaScript functionality to be included in the final documents to enable interactive document features

Generated files:
* `cirodown.js` packaged http://browserify.org[browserify] JavaScript for browser usage, generated with `npm run browserify`

==== `build-sass`
{id=build-sass}

The \a[build-sass] script takes as input the \a[cirodown.scss] https://sass-lang.com[SASS] scss file, and generates various `.css` versions from it required to properly view generated Cirodown HTML:
* `cirodown.min.css`: contains the converted `cirodown.scss` plus online `@import` requests for stuff like \x[mathematics][KaTeX]
* `cirodown.embed.min.css`: like `cirodown.min.css` but embeds all external CSS in-place without any `@import`. Used to distribute standalone pages: \x[html-embed]
* `cirodown.local.min.css`: only includes style generated from `cirodown.scss`, not external styles. Used when the external styles are present locally, e.g. via `node_modules`. I think this was done because https://stackoverflow.com/questions/34139675/css-import-not-working-for-local-file-file-path-is-correct[`@import`] was not working with local flies?

==== Autogenerated tests

The following scripts generate parametrized Cirodown examples that can be used for performance or other types of interactive testing:
* \a[generate-deep-tree]:

  ``
  ./generate-deep-tree 2 5 > deep_tree.tmp.ciro
  ./cirodown deep_tree.tmp.ciro
  ``

  Originally designed to be able to interactively play with a huge \x[table-of-contents] to streamline JavaScript open close interaction.

===== generate-paragraphs

``
./generate-paragraphs 10 > main.ciro
``

Output:
``
0

1

2

3

4

5

6

7

8

9
``

=== Performance

==== `--debug-perf`
{id=debug-perf}

print performance statistics to stderr, for example
``
./cirodown --debug-perf README.ciro
``
could output:
``
perf start: 181.33060800284147
perf tokenize_pre: 181.4424349963665
perf tokenize_post: 318.333980999887
perf parse_start: 319.1866770014167
perf post_process_start: 353.5477180033922
perf post_process_end: 514.1527540013194
perf render_pre: 514.1708239987493
perf render_post: 562.834307000041
perf end: 564.0349840000272
perf convert_input_end 566.1234430000186
perf convert_path_pre_sqlite 566.1564619988203
perf convert_path_pre_sqlite_transaction 566.2528780028224
perf convert_path_post_sqlite_transaction 582.256645001471
perf convert_path_end 582.3469280004501
``

==== Speed comparison to other markup language implementations

One quick and dirty option is to use `generate-paragraphs` which generates output compatible for most markup languages:
``
./generate-paragraphs 100000 > tmp.ciro
``

On Ubuntu 20.04 \a[https://cirosantilli.com/linux-kernel-module-cheat/#p51][Lenovo ThinkPad P51] for example:
* Cirodown 54ba49736323264a5c66aa5d419f8232b4ecf8d0 + 1, Node.js v12.18.1
  ``
  time ./cirodown tmp.ciro
  ``
  outputs:
  ``
  real    0m5.104s
  user    0m6.323s
  sys     0m0.674s
  ``
* Asciidoctor 2.0.10, Ruby 2.6.0p0:
  ``
  cp tmp.ciro tmp.adoc
  time asciidoctor tmp.adoc
  ``
  outputs:
  ``
  real    0m1.911s
  user    0m1.850s
  sys     0m0.060s
  ``
* https://github.com/commonmark/cmark[cmark] 0.29.0:
  ``
  cp tmp.ciro tmp.md
  time cmark tmp.md > tmp.md.html
  ``
  outputs:
  ``
  real    0m0.091s
  user    0m0.070s
  sys     0m0.021s
  ``
  Holy cow, it is 200x faster than Asciidoctor!
* `cat` just to find the absolute floor:
  ``
  time cat tmp.ciro > tmp.tmp
  ``
  outputs:
  ``
  real    0m0.006s
  user    0m0.006s
  sys     0m0.000s
  ``

=== Internals API

Tokenized token stream and AST can be obtained as JSON from the API.

Errors can be obtained as JSON from the API.

Everything that you need to write Cirodown tooling, is present in the main API.

All tooling will be merged into one single repo.

=== The `\Toplevel` implicit macro
{id=toplevel}

Every Cirodown document is implicitly put inside a `\Toplevel` document and:
* any optionally given arguments at the very beginning of the document will be treated as arguments of the `\Toplevel` macro
* anything else will be put inside the `content` argument of the `\Toplevel` macro

E.g., a Cirodown document that contains:
``
{title=My favorite title.}

And now, some content!
``

is morally equivalent to:
``
\Toplevel{title=My favorite title.}
[
And now, some content!
]
``
In terms of HTML, the `\Toplevel` element corresponds to the `<html>`, `<head>`, `<header>` and `<footer>` elements of a document.

Trying to use the `\Toplevel` macro explicitly in a document leads to an error.

==== HTML document title

* if the `title` argument of `toplevel` is given, use that
* otherwise, if the document has a `\H[1]`, use the title of the first such header
* otherwise use a dummy value

=== CSS

Our CSS is located at \a[main.scss] and gets processed through https://sass-lang.com[Sass].

To generate the CSS during development after any changes to that file, you must run:
``
npm sass
``

=== Formal grammar

TODO. Describe Cirodown's formal grammar, and classify it in the grammar hierarchy and parsing complexity.

=== Release procedure

Manually update the \a[package.json] version:
``
vim package.json
``
Create a tag to match, push and release the package:
``
git add -u
t=0.3.1
git commit -m "$t"
git tag -m "$t" "$t"
git push --follow-tags
npm publish
``

=== TODO

* correct ID generation algorithm to be Unicode robust
* local downloads and single HTML page with image / script embeds
* image management: upload to separate media directory
* parse_error: add in document error message, currently only to stderr
* on the AST, put contents of headers inside headers
* auto-toc after the first paragraph

== Related projects

Static wiki generators: this is perhaps the best way of classifying this project :-)
* https://github.com/gollum/gollum[]: already has a local server editor! But no WYSIWYG nor live preview. Git integration by default, so when you save on the UI already generates a Git commit. We could achieve that with: https://github.com/isomorphic-git/isomorphic-git[], would be really nice.
* https://github.com/wcchin/markypydia

Static book generators:
* https://github.com/rstudio/bookdown[], https://bookdown.org/[]. Very similar feature set to what we want!!! Transpiles to markdown, and then goes through Pandoc: https://bookdown.org/yihui/bookdown/pandoc.html[], thus will never run on browser without huge translation layers. But does have an obscene amount of output formats however.
* https://gohugo.io/[Hugo]. Pretty good, similar feature set to ours. But Go based, so hard on browser, and adds adhoc features on top of markdown once again
* https://en.wikipedia.org/wiki/Personal_wiki
  * https://github.com/vimwiki/vimwiki
* https://github.com/hplgit/doconce

Less related but of interest, similar philosophy to what Ciro wants, but no explicitly reusable system:
* http://www.uprtcl.io/
* https://libretexts.org
* https://physics.info/
* https://hypertextbook.com/
* https://tutorial.math.lamar.edu/
